feat(platform): add phase8 real load execution evidence pipeline#22
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f9426e7968
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| "stage": stage, | ||
| "start_time": str(metadata.get("start_time", "")).strip(), | ||
| "end_time": str(metadata.get("end_time", "")).strip(), | ||
| "vu": int(metadata.get("vu", metric(summary, "vus_max", "max", 0))), |
There was a problem hiding this comment.
The merge step currently sets each stage vu from stage metadata first, and those metadata files are generated by the run scripts using the target VU constant rather than an observed value. This allows a run that never actually reached 100k VUs to still produce achieved_concurrency=100000 and pass certification gates, because validation only checks the merged achieved_concurrency field. Use k6-observed metrics (for example vus_max) as the source of truth and treat metadata as informational.
Useful? React with 👍 / 👎.
| "dropped_connections": int(metrics.get("realtime.dropped_connections", 0)), | ||
| "gateway_restarts": int(metrics.get("gateway.restarts", 0)), |
There was a problem hiding this comment.
These fields default to 0 when absent, but the bundled snapshot collector (tools/phase8_collect_prometheus_snapshot.py) does not query realtime.dropped_connections or gateway.restarts, so merged evidence will report zero by default and can incorrectly satisfy threshold checks during real instability. This creates false-positive certifications unless missing critical metrics are treated as an error (or explicitly queried and required).
Useful? React with 👍 / 👎.
| pods = [item for item in snapshot.get("items", []) if item.get("kind") == "Pod"] | ||
| for pod in pods: | ||
| if not pod_ready(pod): | ||
| errors.append(f"Pod/{pod.get('metadata', {}).get('name', 'unknown')} is not ready") |
There was a problem hiding this comment.
The validator marks the snapshot invalid if any pod in the namespace is not Running and ready, including unrelated completed Job/CronJob pods. In a production-like namespace with normal completed maintenance/migration pods, this will block certification even when all required gateway/stateful components are healthy. The readiness loop should scope to the certification workloads (or ignore terminal Succeeded pods) to avoid false failures.
Useful? React with 👍 / 👎.
1 participant
Summary
Adds Phase 8.2 real load execution evidence pipeline so ArchIToken can progress from 100k certification gates to machine-verifiable real execution evidence.
Scope
Validation
Boundary
No real external 100k load run was executed in this PR.
This PR does not claim ArchIToken is certified for 100k concurrent users. It adds the execution evidence pipeline required to run, merge, validate, and certify real external 100k load evidence.
No proprietary RealBIM / BlackHole / OptRapid runtime asset is imported into open-source core.